An Incremental Pattern Matching Algorithm for the Pipelined Lexical Scanner

Lexical scan is a basic part of language processors such as assemblers, compilers, and source code interpreters [2]. The lexical scan takes a large amount of processing time, although the processing algorithm is rather simple and regular. Hence, the hardware realization is especially effective for the speed-up of the processing. Based on the experience of the design and evaluation of a hardware lexical scanner [1,5], an efficient pattern matching algorithm was devised to be incorporated in the pipelined lexical scanner [3]. In this approach, variable-length character strings are converted into compact binary form for the easiness of manipulation in addition to the usual lexical scan. In this paper, details of the lexical scanning algorithm with string compaction are disclosed. The algorithm is based on the associative lexical table and the character stream oriented pattern matching algorithm. The performance evaluation is also given for the actual sample programs of C and PASCAL.